Designing application software in wide area network settings

Progress in methodologies for developing robust local area network software has not been matched by similar results for wide area settings. The design of application software spanning multiple local area environments is examined. For important classes of applications, simple design techniques are presented that yield fault tolerant wide area programs. An implementation of these techniques as a set of tools for use within the ISIS system is described

Cache Serializability: Reducing Inconsistency in Edge Transactions

Read-only caches are widely used in cloud infrastructures to reduce access latency and load on backend databases. Operators view coherent caches as impractical at genuinely large scale and many client-facing caches are updated in an asynchronous manner with best-effort pipelines. Existing solutions that support cache consistency are inapplicable to this scenario since they require a round trip to the database on every cache transaction. Existing incoherent cache technologies are oblivious to transactional data access, even if the backend database supports transactions. We propose T-Cache, a novel caching policy for read-only transactions in which inconsistency is tolerable (won't cause safety violations) but undesirable (has a cost). T-Cache improves cache consistency despite asynchronous and unreliable communication between the cache and the database. We define cache-serializability, a variant of serializability that is suitable for incoherent caches, and prove that with unbounded resources T-Cache implements this new specification. With limited resources, T-Cache allows the system manager to choose a trade-off between performance and consistency. Our evaluation shows that T-Cache detects many inconsistencies with only nominal overhead. We use synthetic workloads to demonstrate the efficacy of T-Cache when data accesses are clustered and its adaptive reaction to workload changes. With workloads based on the real-world topologies, T-Cache detects 43-70% of the inconsistencies and increases the rate of consistent transactions by 33-58%.Comment: Ittay Eyal, Ken Birman, Robbert van Renesse, "Cache Serializability: Reducing Inconsistency in Edge Transactions," Distributed Computing Systems (ICDCS), IEEE 35th International Conference on, June~29 2015--July~2 201

Achieving Reliability Through Distributed Data Flows and Recursive Delegation

Strong reliability properties, such as state machine replication or virtual synchrony, are hard to implement in a scalable manner. They are typically expressed in terms of global membership views. As we argue, global membership is non-scalable. We propose a way of modeling protocols that does not rely on global membership. Our approach is based on the concept of a distributed data flow, a set of messages distributed in space and time. We model protocols as networks of such flows, constructed through recursive delegation. The resulting system uses multiple small membership services instead of a single global one while still supporting stronger properties. Our work was inspired by the functional approach to modeling distributed systems pioneered by I/O automata. This paper focuses on the basic model. Internal details of our system architecture and a compiler that translates protocols from our data flow language to real executable code will be discussed elsewhere.This work was supported in part by grants from AFOSR, AFRL, NSF, and Intel Corporatio

A Brief Overview of the NEBULA Future Internet Architecture

NEBULA is a proposal for a Future Internet Architecture. It is based on the assumptions that: (1) cloud computing will comprise an increasing fraction of the application workload offered to an Internet, and (2) that access to cloud computing resources will demand new architectural features from a network. Features that we have identified include dependability, security, flexibility and extensibility, the entirety of which constitute resilience.NEBULA provides resilient networking services using ultrareliable routers, an extensible control plane and use of multiple paths upon which arbitrary policies may be enforced. We report on a prototype system, Zodiac, that incorporates these latter two features

Implementing High Performance Multicast in a Managed Environment

Component integration environments such as Microsoft .NET and J2EE have become widely popu-lar with application developers, who benefit from standardized memory management, system-wide type checking, debugging, and performance analysis tools that operate across component boundaries. This paper describes QuickSilver Scalable Multicast1 (QSM), a new multicast platform designed to achieve high performance in managed environments. Memory-related overheads and phenomena related to schedul-ing are shown to dominate the behavior of the system. We discuss techniques that helped us to alleviate these problems, and argue that they reveal general principles applicable to other kinds of high-data-rate protocols and applications in managed settings

Optimizing Buffer Management for Reliable Multicast

Reliable multicast delivery requires that a multicast message be received by all members in a group. Hence certain or all members need to buffer messages for possible retransmissions. Designing an efficient buffer management algorithm is challenging in large multicast groups where no member has complete group membership information and the delivery latency to different members could differ by orders of magnitude. We propose an innovative two-phase buffering algorithm, which explicitly addresses variations in delivery latency seen in large multicast groups. The algorithm effectively reduces buffer requirements by adaptively allocating buffer space to messages most needed in the system and by spreading the load of buffering among all members in the group. Simulation results demonstrate that the algorithm has good performance

Scalable Publish-Subscribe in a Managed Framework

Reliable multicast, publish-subscribe and group communication are highly effective in support of replication and event notification, and could serve as the enabling technologies for new types of applications that are both interac-tive and decentralized. To fully realize this vision, we need a high-performance, scalable, and reliable multicast en-gine as an integral part of the runtime environment. Since the majority of development today is done in managed, strongly-typed environments such as Java or .NET, integration with such environments is of particular importance. What factors limit performance and scalability of a reliable multicast engine in a managed environment? What support from the runtime could improve performance, avoid instabilities, or make such systems easier to build? This paper sheds light on these questions by analyzing the performance of QuickSilver Scalable Multicast (QSM), a new multicast protocol and system built entirely in .NET. Memory-related overheads and scheduling-related phenomena are shown to dominate the behavior of our system. We discuss techniques that helped us alleviate some of these problems, and point to areas where better support from the runtime would be desirable

A Randomized Error Recovery Algorithm for Reliable Multicast

An efficient error recovery algorithm is essential for reliable multicast in large groups. Tree-based protocols (RMTP, TMTP, LBRRM) group receivers into local regions and select a repair server for performing error recovery in each region. Hence a single server bears the entire responsibility of error recovery for a region. In addition, the deployment of repair servers requires topological information of the underlying multicast tree, which is generally not available at the transport layer. This paper presents RRMP, a randomized reliable multicast protocol which improves the robustness of tree-based protocols by diffusing the responsibility of error recovery among all members in a group. The protocol works well within the existing IP multicast framework and does not require additional support from routers. Both analysis and simulation results show that the performance penalty due to randomization is low and can be tuned according to application requirements

The Power of Indirection: Achieving Multicast Scalability by Mapping Groups to Regional Underlays

Reliable multicast is a powerful primitive, useful for data replication, event notification (publish-subscribe), fault tolerance and other purposes. Yet many of the most interesting applications give rise to huge numbers of heavily overlapping groups, some of which may be large. Existing multicast systems scale scale poorly in one or both respects. We propose the QuickSilver Scalable Multicast protocol (QSM), a novel solution that delivers performance almost independent of the number of groups and introduces newmechanisms that scale well in the number of nodes with minimal performance and delay penalties when loss occurs. Key to the solution is a level of indirection: a mapping of groups to regions of group overlap in which communication associated with different protocols can be merged. The core of QSM is a new regional multicast protocol that offers scalability and performance benefits over a wide range of region sizes